Adjustment for output image of image data

ABSTRACT

The present invention is an image processing apparatus for processing an image data using an image file which includes the image data and image processing control information to be used for processing the image data. The apparatus comprises an automatic picture quality adjuster configured to automatically regulate lightness of the image data according to lightness of a whole image expressed by the image data and an adjustment degree determiner configured to determine a degree of the auto adjustment based on the image processing control information.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.10/520,858, filed on Jan. 10, 2005, the disclosure of which is herebyincorporated by reference in its entirety for all purposes.

TECHNICAL FIELD

The present invention relates to an image adjustment technique thatadjusts the lightness of image data.

BACKGROUND ART

The picture quality of image data generated by a digital still camera(DSC), a scanner, or other image input devices is often subjected toautomatic adjustment on a personal computer or a printer. Parametersrelating to adjustment of the picture quality include brightness(lightness) of an image. The brightness of the image is regulated, suchthat the reflectivity of an image corresponding to the brightness ismade approximate to the reflectivity of a standard light reflector (forexample, a reflector having a reflectivity of 18%), according to thebrightness over the whole image expressed by the image data. Suchregulation prevents the whole image from being excessively darkened orbrightened.

In some cases, the user may desire adequate expression of the brightnessnot over the whole image but in only a part of the image. For example,in the case of portraits, emphasis is given not on the brightness overthe whole image but on the brightness of only a figure, which is themain subject. In such cases, regulation according to the brightness overthe whole image may cause the figure from being excessively brightened.

DISCLOSURE OF THE INVENTION

The object of the present invention is thus to solve the problems of theprior art technique discussed above and to provide a technique ofpreventing automatic adjustment of lightness of an image against theuser's intention for photographing.

In order to attain at least part of the above and the other relatedobjects, a first application of the present invention is directed to animage processing apparatus for processing an image data using an imagefile. The image file includes the image data and image processingcontrol information to be used for processing the image data. Theapparatus comprises an automatic picture quality adjuster and aregulation degree determiner. The automatic picture quality adjuster isconfigured to automatically adjust lightness of the image data accordingto lightness of a whole image expressed by the image data. Theadjustment degree determiner is configured to determine a degree of theauto adjustment based on the image processing control information.

The first application of the invention determines the degree of the autoadjustment of the lightness over the whole image expressed by the imagedata, based on the image processing control information included in theimage file. For example, when the user intends to express the brightnessadequately not over the whole image but in only a part of the image,this arrangement desirably reduce forcible auto adjustment against theuser's intention.

In one preferable embodiment of the image processing apparatus, theimage processing control information includes light metering informationshowing whether a specific metering method is used for photographing.The specific metering method is measuring light only in a specifiedfield of view that is a part of the image. The adjustment degreedeterminer is configured to reduce the degree of the auto adjustmentwhen the light metering information shows that the specific meteringmethod is applied for photographing.

When the specific metering method that measures light only in thespecified field of view is applied for photographing, it is expectedthat the user intends to adequately express the lightness in thespecified field of view. The arrangement of this embodiment desirablyprevents forcible auto adjustment against the user's intention forphotographing. The terminology ‘reducing the degree of auto adjustment’includes reduction of the auto adjustment degree to zero, that is,complete prohibition of the auto adjustment.

In the image processing device of the invention, it is preferable thatthe light metering information represents a selected method of lightmetering among a plurality of metering methods including averagedmetering, center-weighted metering, spot metering, multi-spot metering,divisional light metering, and partial light metering, and the specificmetering methods include the spot metering, the multi-spot metering, andthe partial light metering.

In one preferable application of the image processing apparatus, theadjustment degree determiner is configured to provide a user with a userinterface to allow the user to select the degree of the auto adjustmentwhen the image processing control information indicates thecenter-weighted metering. The terminology ‘selecting the degree of autoadjustment’ includes an option of setting ‘0’ to the auto adjustmentdegree, that is, complete prohibition of the auto adjustment.

The center-weighted metering with a greater degree of emphasis is closerto the partial light metering, while the center-weighted metering with aless degree of emphasis is closer to the divisional light metering. Thearrangement of providing the user interface that allows the user toselect the degree of the auto adjustment of the auto adjustment ensuresthe image processing suitable for the user's intention forphotographing.

A second application of the invention is directed to an image processingapparatus for processing an image data using an image file. The imagefile includes the image data and image processing control information tobe used for processing the image data. The apparatus comprises anautomatic picture quality adjuster and an adjustment mode selector. Theautomatic picture quality adjuster is configured to automatically adjustlightness of the image data according to lightness of a whole imageexpressed by the image data in one of a plurality of adjustment modes.The adjustment mode selector is configured to select one of theplurality of adjustment modes according to the image processing controlinformation. The plurality of adjustment modes includes a plurality ofadjustment modes having a difference in a degree of lightnessadjustment. Here the multiple adjustment modes include plural adjustmentmodes having a difference in a degree of lightness correction.

The present invention is also directed to an image output apparatus foroutputting image data in response to an image file, which includes theimage data and image processing control information used for imageprocessing of the image data. The image output device of the inventionincludes the image processing device having any of the arrangementsdiscussed above, and an image output unit that outputs an imageaccording to the image-processed image data.

The technique of the present invention is actualized by a variety ofapplications, which include image file generation devices, image outputdevices, image processing methods, computer programs that attain thefunctions of such devices and methods, recording media in which suchcomputer programs are recorded, and data signals that include suchcomputer programs and are embodied in carrier waves.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an image processing system in one embodiment of theinvention;

FIG. 2 is a block diagram schematically illustrating the structure of adigital still camera working as an input device of generating imagedata;

FIG. 3 is a block diagram schematically showing the configuration of acomputer PC and a color printer as an output device of outputting imagedata;

FIG. 4 schematically shows the structure of an image file GF in theembodiment of the invention;

FIGS. 5(A) and 5(B) show one example of attribute information stored inExif IFD of the image file GF;

FIG. 6 is a flowchart showing an image processing routine executed bythe computer PC;

FIG. 7 is a flowchart showing a processing routine of auto picturequality adjustment process executed in the embodiment;

FIGS. 8(A) and 8(B) show the specifications of auto lightness correctionexecuted in the embodiment; and

FIG. 9 shows a user interface that allows a user to select execution orprohibition of auto correction.

BEST MODE FOR CARRYING OUT THE INVENTION

One mode of carrying out the invention is discussed below as a preferredembodiment in the following sequence:

-   A. Construction of Image Processing System-   B. Structure of Image File-   C. Image Processing by Computer PC-   D. Modifications

A. Construction of Image Processing System

FIG. 1 illustrates an image processing system 10 in one embodiment ofthe invention. The image processing system 10 includes a digital stillcamera 12 functioning as an input device that generates original imagedata, a personal computer PC functioning as an image processing devicethat performs image processing of the original image data generated bythe digital still camera 12, and a color printer 20 functioning as anoutput device that outputs processed images.

The digital still camera 12, the personal computer PC, and the colorprinter 20 are mutually connectable via a cable CV. In the state ofconnection by the cable CV, the digital still camera 12 and the otherconstituents are allowed to transmit and receive image files via thecable CV. In the status of no connection by the cable CV, the digitalstill camera 12 and the other constituents are allowed to transmit imagefiles therebetween via a memory card MC.

FIG. 2 is a block diagram schematically illustrating the structure ofthe digital still camera 12 working as the input device of generatingimage data. The digital still camera 12 focuses an image through anoptical lens on a charge coupled device (CCD) so as to electricallyrecord a still image.

The digital still camera 12 includes an optical circuit 121 having a CCDthat converts optical signals into electrical signals, an imageacquisition circuit 122 that controls the optical circuit 121 to acquireimage data, an image processing circuit 123 that processes the acquiredimage data, and a control circuit 124 that controls these respectivecircuits. The digital still camera 12 further has a selection/decisionbutton 126 as a user interface and a liquid crystal display 127 used forpreview of photographed images and user interfaces.

The photographing process of the digital still camera 12 (that is, theprocess of acquiring image data) includes the steps of (a) the user'ssetting of a photographing mode, (2) photographing (input of imagedata), (3) image processing, and (4) recording of an image file, whichare carried out sequentially. The user's setting of the photographingmode includes selection of a method of light metering. Available optionsof the light metering include divisional light metering and spot lightmetering.

The divisional light metering divides a finder plane corresponding to animage area into multiple sections, measures the light in these multiplesections, and computes an adequate exposure based on information on themultiple sections. The spot light metering measures the light in only aspot area (for example, an area in the vicinity of the center of thefinder plane) and computes an adequate exposure.

The user checks the display contents on the liquid crystal display 127and manipulates the selection/decision button 126 to select a desiredmethod of light metering. On selection of the light metering, theadequate exposure is computed in real time, and appropriate settings ofan aperture and a shutter speed are determined according to the computedexposure.

The user presses a shutter button to effectuate photographing. Inresponse to a press of the shutter button, photographing is performedwith the settings of the aperture and the shutter speed determined inthe above manner. When the face of a figure as a main subject is locatedin the spot area in the selected spot light metering, the settings ofthe aperture and the shutter speed are determined on the basis of thebrightness of the face of the figure. This arrangement performsphotographing with the exposure optimized not for the whole finder planebut for the main subject to generate original image data.

The original image data thus generated are subjected to image processingfor storage. This image processing is a pre-treatment of the originalimage data, prior to storage into the memory card MC. The generalprocedure converts the original image data into a JPEG format suitablefor storage of photographic images. After conversion into the JPEGformat, the procedure adds photographing information PI to the convertedimage data to create an image file.

The photographing information PI regards photographing conditions andincludes information representing the selected light metering. Theprocess of acquiring image data in the digital still camera 12terminates with recording of the image file into the memory card MC. Thestructure of the image file will be discussed later.

FIG. 3 is a block diagram schematically showing the configuration of thecomputer PC and the color printer 20 as the output device of outputtingimage data. The computer PC has a slot 22 for reading an image file fromthe memory card MC inserted therein and a print data generation circuit23 for generating print data, according to which the color printer 20carries out printing. The print data generation circuit 23 includes acentral processing unit (CPU) 231 that executes arithmetic operationsfor generation of print data, a hard disk 232 that stores programsexecuted by the CPU 231, results of the arithmetic operations by the CPU231, and other data, and a random access memory (RAM) 233 thattemporarily stores these programs and data.

The color printer 20 is capable of outputting color images. One typicalexample of the color printer 20 is an ink jet printer that ejects fourdifferent color inks, cyan (C), magenta (M), yellow (Y), and black (K)on a printing medium to form a dot pattern and thereby complete aprinted image.

B. Structure of Image File

FIG. 4 schematically shows the structure of an image file GF in theembodiment of the invention. The image file GF has a file structure inconformity with Exchangeable image file format (Exif) for digital stillcameras. This format is specified by Japan Electronics and InformationTechnology Industries Association (JEITA). According to this format,JPEG-Exif files that store compressed JPEG data as image data areincluded in Exif files (files of the Exif format).

The image file GF includes an SOI marker segment 101 representing aheader of compressed data, an APP1 marker segment 102 storing attributeinformation of the Exif format, an APP2 marker segment 103 storing Exifextended data, a DQT marker segment 104 defining a quantization table, aDHT marker segment 105 defining a Huffman table, a DRI marker segment106 defining an insertion interval of a restart marker, an SOF markersegment 107 representing various parameters relating to frames, an SOSmarker segment 108 representing various parameters relating to scanning,an EOI marker segment 109 representing a termination of the compresseddata, and an image data storage area 110.

The APP1 marker segment 102 stores an APP1 marker 1021, an Exifidentification code 1022, a TIFF header and other attribute information1023, and thumbnail image data 1024. The attribute information 1023 hasa TIFF structure with a file header (TIFF header) and includes, in thecase of an Exif-JPEG file, 0^(th) IFD storing attribute informationrelating to compressed image data, Exif IFD storing the photographinginformation PI and other attribute information inherent to the Exifformat and 1^(st) IFD storing attribute information relating tothumbnail images. The Exif IFD is pointed by an offset from the TIFFheader stored in the 0^(th) IFD. Tags for identifying various pieces ofinformation are used in the IFD, and the respective pieces ofinformation may be referred to as tag names.

FIG. 5 shows one example of the attribute information stored in the ExifIFD of the image file GF. FIG. 5(A) shows the structure of the attributeinformation stored in the Exif IFD. The attribute information includesvarious tags including tags relating to a version and tags relating tophotographing conditions. The tags relating to the photographingconditions store settings of exposure time, F number of a lens, ISOsensitivity, shutter speed, exposure, luminance, light metering, andother parameters according to specified offsets as the photographinginformation PI. The photographing information PI may be recorded in thedigital still camera 12 in the photographing process, as describedpreviously.

FIG. 5(B) shows values of the data stored in the light metering segmentand their meanings. For example, when the spot light metering isselected for photographing with the digital still camera 12, a value ‘3’is written in the process of recording an image file. In anotherexample, a value ‘5’ is recorded in the case of selection of thedivisional light metering.

Among the methods of light metering shown in FIG. 5(B), the spot lightmetering, the multi-spot light metering, and the partial light meteringmeasure the light in only a specific field of view that is a part of aprinted image. The other methods of light metering measure the lightover the whole visual field. The former three methods of light meteringcorrespond to the ‘specific metering method’ in the terminology of theclaims.

C. Image Processing by Computer PC

FIG. 6 is a flowchart showing an image processing routine executed bythe computer PC. At step S100, the CPU 231 reads the image file GF fromthe memory card MC inserted in the slot 22 and stores the image file GFinto the RAM 233 included in the print data generation circuit 23. Theimage file GF stores image data of the JPEG file format as image dataGD. The image data of the JPEG file format are constructed as compressedYCbCr data.

At step S110, the CPU 231 decompresses the compressed YCbCr data andcarries out color conversion. The color conversion converts the YCbCrdata into RGB data. Conversion into the RGB data is because the RGB datais the subject of image processing executed by the personal computer PCor the color printer 20.

At step S120, the CPU 231 makes the RGB data subjected to a picturequality adjustment process with reference values. The picture qualityadjustment process with reference values adjusts the picture qualitywith desired standard values of parameters (reference values), whichhave been stored in advance in the hard disk 232. Such a picture qualityadjustment process is an image processing technique and is generallyreferred to as auto picture quality adjustment process. The parametersadjusted by the auto picture quality adjustment process includelightness. The details of the auto picture quality adjustment processwith regard to the lightness will be discussed later.

At step S130, the CPU 231 carries out color conversion to convert theimage data (RGB data) undergoing the auto picture quality adjustmentprocess into CMYK data. The CMYK color space is expressible with fourcolor inks, cyan (C), magenta (M), yellow (Y), and black (K) availablein the color printer 20. This color conversion process utilizes a lookuptable, which represents a mapping of the RGB color system to the CMYKcolor system and is stored in the hard disk 232.

At step S140, the CPU 231 carries out a print output process of theresulting image data (CMYK data). In the print output process, the CPU231 performs a halftoning process to generate print data including dotdata, which represents the formation state of dots of the respectivecolor inks, and transmits the generated print data to the color printer20. This terminates the image processing routine.

FIG. 7 is a flowchart showing the details of the auto picture qualityadjustment process executed in this embodiment. At step S200, the CPU231 reads the photographing information PI (see FIGS. 5(A) and 5(B))stored in the Exif IFD of the image file GF. The photographinginformation PI includes a piece of information representing selection ofeither the divisional light metering or the spot light metering forphotographing.

At step S210, the CPU 231 determines whether the selected light meteringfor photographing is the spot light metering, according to the read-outinformation. When the selected light metering is not the spot lightmetering, the program proceeds to step S220 to carry out auto adjustmentincluding auto lightness correction. When the selected light metering isthe spot light metering, on the other hand, the program proceeds to stepS230 to carry out auto adjustment excluding auto lightness correction.

At step S220, the CPU 231 performs auto adjustment of the picturequality including the auto lightness correction. The auto lightnesscorrection automatically lowers the lightness of an image reproduced ona printing medium in the event of an inadequate exposure forphotographing, for example, in the case of an excessively bright imageof the image data GD.

FIG. 8 shows the specifications of the auto lightness correctionexecuted in this embodiment. FIG. 8(A) shows a tone curve used forcorrecting an underexposure, and FIG. 8(B) shows a tone curve used forcorrecting an overexposure. The input level represents values of imagedata before the auto lightness correction, and the output levelrepresents values of image data after the auto lightness correction.

The auto lightness correction varies the shape of the tone curve toregulate the lightness of a printed image. This correction is performedto make the reflectivity of the whole printed image reproduced on aprinting medium approximate to the reflectivity of a standard lightreflector (for example, a reflector having a reflectivity of 18%). Forexample, in the case of an underexposure shown in FIG. 8(A), thereflectivity of the whole image is smaller than the reflectivity of thestandard light reflector. The shape of the tone curve is accordinglyvaried to be upward convex and make the output level higher than theinput level. Such variation of the tone curve makes the reflectivity ofthe whole printed image approach to the reflectivity of the standardlight reflector.

The auto lightness correction, however, has a weak point. In some cases,there is a requirement of adjusting the exposure only at a specifiedpoint on the image plane, for example, when a subject of photographingstands in a pin spot on a stage. Under such circumstances, the spotlight metering, which determines the exposure based on the result oflight measurement only in a specific area that is several percentage ofthe whole image plane, is adopted for photographing to generate imagedata as intended. The auto lightness correction automatically regulatesthe lightness on the basis of the whole image plane to give correctedimage data against the user's intention for photographing.

The processing routine of the embodiment, however, determines whetherthe selected light metering for photographing is the spot light meteringand carries out the auto adjustment excluding the auto lightnesscorrection in the setting of the spot light metering (step S230) asdiscussed above. This arrangement desirably prevents the automaticcorrection of the lightness against the user's intention forphotographing.

As described above, the system of the embodiment is arranged to avoidthe auto lightness correction in the setting of the spot light meteringselected for photographing, in response to the photographing informationPI. This arrangement effectively prevents automatic correction of thelightness of printed images against the user's intention forphotographing, which requires adjustment of the exposure only at aspecified point on the image plane.

The CPU 231 functions as the ‘automatic picture quality adjuster’ and'adjustment degree determiner' in the claims. The photographinginformation PI is included in the ‘image processing control information’in the terminology of the claims.

D. Modifications

The embodiment and its applications discussed above are to be consideredin all aspects as illustrative and not restrictive. There may be manymodifications, changes, and alterations without departing from the scopeor spirit of the main characteristics of the present invention. Someexamples of possible modification are given below.

D-1. In the system of the above embodiment, the digital still camera 12has the two available methods of light metering, that is, the divisionallight metering and the spot light metering, and the auto lightnessadjustment is not performed in the setting of the spot light meteringselected for photographing. One possible application may also prohibitthe auto lightness adjustment in the setting of multi-spot lightmetering or partial light metering selected for photographing withanother digital still camera.

Another possible application may not prohibit auto adjustment but reducethe degree of auto adjustment. The degree of auto adjustment may bereduced by using a modified tone curve with reduced adjustment degree.

The modified tone curve with reduced adjustment degree may be obtained,for example, by giving adequate weights to the two tone curves (FIG. 8)used for auto adjustment and overlapping the weighted tone curves or bygiving adequate weight to a linear tone curve and either of the two tonecurves (FIG. 8) and overlapping the weighted tone curves.

The technique of the present invention is generally constructed toreduce the degree of auto adjustment in the setting of a specific lightmetering selected for photographing to measure the light in only aspecified visual field area corresponding to part of a printed image.

It is preferable that methods of light metering recognizable by theimage processing device based on the information of image files includeaveraged light metering, center emphasis light metering, spot lightmetering, multi-spot light metering, divisional light metering, andpartial light metering. The technique of the invention is thenadvantageously applicable for image processing of image files inconformity with the specifications of Exif Ver. 2.2 and the laterversions, which expect these methods of light metering.

D-2. The procedure of the above embodiment selects either of the autoadjustment including auto lightness adjustment or the auto adjustmentexcluding auto lightness adjustment according to the photographinginformation PI. One possible modification may select the auto adjustmentincluding auto lightness adjustment only in the case of selection of aphotographing mode to compute the adequate exposure based on thebrightness of the whole image area, for example, in the setting ofaveraged light metering or divisional light metering. The technique ofthe present invention is generally constructed to prevent auto lightnessadjustment of printed images in the setting of partial light meteringselected for photographing.D-3. The procedure of the above embodiment automatically determinesexecution or prohibition of auto lightness adjustment according to thephotographing information PI. One modification may ask the user toselect the degree of the auto adjustment, for example, in the case ofselection of the center-weighted metering. The center-weighted meteringis used for photographing with emphasis on the exposure in the center ofthe finder, for example, when a main subject is located in to the centerof the image plane as in the case of a portrait. The center-weightedmetering measures the light, for example, with a degree of emphasis ofapproximately 60% on a circle occupying an area of about 10% in thecenter of the finder.

The center-weighted metering with a greater degree of emphasis is closerto the partial light metering, while the center-weighted metering with aless degree of emphasis is closer to the divisional light metering. Onepreferable application provides the user with a user interface (FIG. 9)to allow the user to select the degree of the auto adjustment. Thisarrangement advantageously ensures image processing desired by the user.The user interface preferably has options of printing multiple imageswith a variety of degree of the auto adjustment.

D-4. The procedure of the above embodiment prohibits auto lightnessadjustment of printed images in the setting of partial light meteringselected for photographing. One modified procedure may receive a manualinput of an auto lightness adjustment prohibition command afterphotographing and determine to allow or prohibit auto lightnessadjustment in response to the command. The technique of the invention isgenerally constructed to determine execution or prohibition of autolightness adjustment according to image processing control information.This arrangement flexibly prevents forcible auto lightness adjustment ofprinted images against the user's intention for photographing.D-5. The procedure of the above embodiment determines execution orprohibition of auto lightness adjustment according to the imageprocessing control information. One modified arrangement may select thedegree of auto lightness adjustment according to the image processingcontrol information. One concrete procedure may provide multipleadjustment modes, which includes a high adjustment mode to make arelatively high degree of adjustment and a low adjustment mode to make arelatively low degree of adjustment, and select one of the multiplecorrection modes according to the image processing control informationto automatically regulate the lightness of image data. In general, thepresent invention may be configured to determine the degree of the autoadjustment based on the image processing control information. The‘reducing the degree of auto adjustment’ includes reduction of the autoadjustment degree to zero, that is, complete prohibition of the autoadjustment.D-6. In the system of the above embodiment, the personal computerfunctions as the image processing device. The color printer or thedigital still camera may have the functions of the image processingdevice. The technique of the invention is not restricted to colorprinting but is also applied to monochromatic printing.D-7. In the system of the above embodiment, the ink jet color printer isused as the output device. The technique of the invention is alsoapplicable to other output devices that can display images, for example,CRT displays, LCD displays, monitors, and projectors. Namely theinvention is not restricted to printed images but is applicable forgeneral images.

When part or all of the functions of the invention are actualized by thesoftware configuration, the software (computer programs) may be providedin a form stored in a computer readable recording medium. The ‘computerreadable recording medium’ of this invention is not limited to portablerecording media, such as flexible disks and CD-ROMs, but also includesinternal storage devices incorporated in computers like various RAMs andROMs, as well as external storage devices fixed to computers like harddisks.

1. A camera apparatus comprising: a display that displays a photographicsubject; and an image acquisition circuit that determines aphotographing condition based on brightness of a face of a figuredisplayed as the photographic subject on the display, to acquire anoriginal image data photographed under the determined photographingcondition.
 2. The camera apparatus according to claim 1, wherein thephotographing condition includes at least one of an aperture, a shutterspeed, and an exposure.
 3. The camera apparatus according to claim 2,further comprising an image processing circuit that converts theoriginal image data into a JPEG format.
 4. The camera apparatusaccording to claim 3, wherein the photographing condition is determinedwhen a photographing mode is selected.
 5. The camera apparatus accordingto claim 4, wherein the photographing mode includes a light meteringmethod.
 6. The camera apparatus according to claim 5, wherein the lightmetering method includes a spot light metering method that measureslight in only a vicinity of a center of the display.
 7. A method forcontrolling a camera, comprising: displaying a photographic subject on adisplay of the camera; determining a photographing condition based onbrightness of a face of a figure displayed as the photograph subject onthe display, and generating an original image data photographed underthe determined photographing condition.
 8. The method according to claim7, wherein the photographing condition includes at least one of anaperture, a shutter speed, and an exposure.